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Abstract 



This research examines the validity and reliability of aggregating 
individuals' perceptions of organizational (college and 
university) characteristics to more macro units of analysis. The 
results suggest that commonly employed aggregation practices may 
lead to spurious conclusions. In addition, they suggest that the 
reliability and validity of previous studies In higher education 
that draw conclusions based on aggregated perceptual data may be 
seriously flawed. 
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The Reliability and Validity of Organizational Climates 



Introduction 

The use of perceptual data to assess organizational conditions and 
practices in higher education is commonplace. Instrument:^ such as 
the Institutional Goals Inventory (Educational Testing Service) , 
Institutional Functioning Inventory (ETS), and the Higher 
Education Management Institute's Needs Assessment Survey — all of 
which assess individuals ' perceptions of various institutional 
characteristics — have been used by hundreds of institutions for 
purposes of self-studyi planningi evaluation, and polic^onaking. 
This type of data is also frequently used by researchers to 
investigate the relationship between various organizational 
chararacteristics and practices in colleges and universities. 

More often than not/ individuals' perceptions of an organization's 
characteristics are aggregated to more macro units of analysis — 
for example/ to the department or institution level. The utility 
of the resulting index (the mean) rests on the assumption that it 
reflects how each member of the group generally perceives/ imputes 
meaning/ and responds to the environment- However/ previous 
research indicates that this asstunption is generally unwarranted. 

Unfortunately/ very little research has been done on the 
appropriateness of aggregating individuals ' perceptions of 
organizational characteristics in colleges and universities. 
Failure to address this issue may account for many of the faulty 
policy-decisions and inconclusive rer'^arch studies that are 
documented in the literature. This paper reports the results of a 
study concerned with assessing the reliability and validity of 
aggregating individuals ' perceptions of organizational 
characterisitics in colleges and universities. 



Theoretical Background 

The literature in psychology refers to individuals* perceptions of 
organizational characteristics as "psychological climates" (Joyce 
and Slccum/ 1979; Weick/ 1979; James and JoneS/ 1974; Jones and 
James, 1979; Schneider/ 1975/ 1981; Gavin and Howe, 1975; Payne/ 
Pineman/ and Wall/ 1976; Woodman and King, 1978), It is generally 
assumed that psychological climates are influenced by individuals' 
experiences, biases, preferences — that is, they are viewed as 
subjective and psychological in nature. Such perceptions are 
useful because they help us understand the influence of the 
organizational environment on individual performance and 
satisfaction. 

When psychological climate data are aggregated the resulting 
measurf^s are referred to as "organizational climates" (Drexler, 
1977; /oyce and Slocum/ 1979; Gavin, 1975; Gavin and Kelley/ 1978; 
Howe, 1977? Jones and James, 1979; Newman, 1975), The utility of 
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organizational climates rests on the assumption that they reflect 
how each member of the group generally perceives, imputes meaning* 
and responds to the environment. However* a body of research that 
has evolved during the last fifteen years indicates that when 
aggregate indices are based on existing groups, both the 
reliability and the validity of the resulting index are suspect. 

Current thinking in climate research suggests that the unit of 
theory for climate, including organizational climate* is the 
individual. This thinking is based on the view that climate 
involves a set of macro perceptions that reflect how environments 
are cognitively represented in terms of their psychological 
meaning and significance to the individual. However, as noted by 
James (1982), characterizing the "unit of theory" for climate as 
the individual does not imply that climate perceptions cannot be 
aggregated and used to describe organizations (or groups, 
divisions, departments, etc. ) . 

There are, however, criteria to be met before aggregating scores 
on a variable for which the "unit of theory" is the individual. 
These criteria include (Jo^ce and Slocum, 1984, p. 722): (1) dis- 
crimination, or demonstrable differences between the mean climate 
perceptions of different groups (Djexler, 1977; Howe, 1977; 
Newman, 1975) ; (2) predictable relationships to organizational or 
individual performance (Pritchard and Karasick, 1973); and 
(3) internal consistency, or agreement of perceptions within 
groups (Howe, 1977) * 

Most of the research that has been done on the reliability and 
validity of aggregating individual perceptual data to more macro 
units of analysis has focused on business and military organiza- 
tions (Drexler, 1977; James and Sells, 1981; Jones and James, 
1979; Joyce and Slocum, 1979; Payne and Pugh, 1976; Powell and 
Butterfield, 1978). Review of these studies provides little 
support for the validity of aggregating individuals' perceptions 
of organizational characteristics. Estimates of perceptual 
aggreement (based on intraclass correlation coefficients) ranged 
between .00 and .50, with a median of approximately .12 (Hater, 
1977; James and Sells, 1981; Jones and James, 1979). 

Little if any research has been done on the reliability and 
validity of organizational climates in higher education. A search 
of the Educational Resources Information Center (ERIC) database 
produced only three articles since 1966 that dealt with the 
validity of organizational climates in education. A search of 
Dialog's "Psychinfo" database — which includes descriptions of all 
published articles in psychology since 1967 — produced a similiar 
number. Even the technical manuals for instruments frequently 
used to assess perceptions of college and university conditions — 
the Institutional Functioning Inventory (Educational Testing 
Service), the Institutional Goals Inventory (Educational Testing 
Sarvice) , and the Higher Education Management Institute's Meeds 
Assessment Survey- — make no mention of the reliability or validity 
of aggregate climate data, even though aggregate statistics form 
the basis for their feedback reports. 



Rellalolllty and the intraclass Correlation 



Reliability may be defined as the ratio of true score variance to 
total score variance (Winer* 1971 ) ; 

where '^"^ equals true score variance, and equals error score 

variance, m terms of this definition of reliability, it is 
easily shown that the Intraclass correlation provides a measure of 
the reliability of measurements (Winer* 1971; Bartko, 1966; 1976; 
Ebel, 1967) . 

Derived In terms of a single factor analysis of variance model 
with repeated measures (Winer, 1971), the formula for the 
Intraclass correlation coefficient for k raters is 

ICC(k) = MSb. people - "Sw. people (2) 

"Sb. people 

where, iC = number of raters 

MSb. people ^ mean square between people 
MSw. people = mean square within people 

Negative coefficients are defined as zero. The reliability of a 
single measurement Is given by 

ICC(l) = MSb, people - "Sw. people (3) 
MSb, people + t^-D MS„, people 

ICC(l) Is most directly interpreted as the average correlation 
between any two judges ratings. it Is easily shown that the 
magnitude of ICC(l) Is a function of Intra-class rater agreement. 

ICC(k) can be estimated from ICC(l) by application of the 
Spearman-Brown prophecy formula: 



ICC(k) = k*ICC(l ) (4) 

1 + (k-l)*lCC(l) 



ICC(k) reflects the correlation between the average ratings of two 
randomly selected groups of judges* However, James (1982) 
demonstrates that ICC(k) Is not a measure of perceptual agreement. 
Using the Spearman-Brown prophecy formula, Ja^ies demonstrates 
that a trivial ICC(l) may lead to a very large ICC(k) . 
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In typical rater reliability studies, a random sample of n targets 
is rated independently by k judges. Three cases of this kind can 
be defined (Shrout and Fleiss, 1979, p. 420); 

1. Each target is rated by a different set of k judges, 
randomly selected from a larger population of judges, 

2. A random sample of k judges is selected from a larger 
population, and each judge rates each target, that is, 
each judge rates n targets altogether. 

3. Each target is rated by each of the same k judges, who 
are the only judges of interest. 



The case 1 model is illustrated below: 

Target Ratings 

1 2 3 3 4 

2 4 5 5 7 8 9 

3 3 4 5 
* « . 

n k 



The case 1 model reflects a situation where the ratings are 
incomplete, and the sources of ratings are unknown. Different 
groups of judges rates associated with with their class. 
Qualitative differences among targets are irrelevant in this 
model. However, it is generally presumed that all targets are 
being rated on the same dimension. In this situation, ICC{1) 
provides us with a measure of average within-class agreement. 

Estimating reliability in case 1 situation presents a special 
problem. The formula requires a value of k, the number of ratings 
of each target. However, case 1 allows for a different number of 
raters for each target. Snedecor (1946), and Ebel (1967) suggest 
that we can obtain an average k which provides a consistent 
estimate of ICC(l) with the following formula; 



ko = 1 [ ^^k -4k£, ] (4) 

n - 1 k 

The case 2 model is illustrated below; 

Target Ratings 

1 1 2 3 ... k 

2 1 2 3 ... k 

3 1 2 3 ... k 
* . 

n 1 2 3 . . . k 



The case 2 model applies when a random sample of raters is 
selected from a larger population, and each rater rates all n 
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targets of interest on a single dimension. In this instance, 
ICC(l) provides a measure of perceptual agreement between raters 
on all targets* The case 3 model differs from the case 2 model 
only in so far that the k raters represent the only raters of 
interest' rather than a random selection* 

In all three models, a high ICC(l) follows from high intra-class 
perceptual agreement* The difference between inter-class and 
intra-class agreement is illustrated below for a case 1 situation. 
In this example/ the ratings and mean rating of two schools on a 
single dimension (e*g., morale) are identical* 

It£m #1 ^ Ratings Mean 

School 1 1,2,3,4,5 3.0 

School 2 1,2,3,4,5 3*0 

In this example ICC(l) equals zero because intra-school agreement 
is zero* In terms of the model specified in equation 3, we 
obtain: 

ICC(l) = MSb^peopie - MSy^^peppie 

MSb, people + (k-D MS^, people 

* 0 - 2*5 = - *25 

0 + (5-1) (2*5) 

= 0 

Coversely, because the intra-class correlation measures within^ 
class agreement, raters in different schools could view their 
schools in entirely opposite but consistent ways, and ICC(l) would 
equal 1*00* This is illustrated below; 

Item #1 Ratings Mean 

School 1 1,1,1,1,1 1*0 

School 2 5,5,5,5,5 5.0 

In terms of the model specified in equation 3 we obtain^ 

ICC(l) «= MSb^ people - MSy^^eoPle 

WSb, people + (k-1) MSw, people 



4^0 



4 + (5-l)*0 
* 1 ,0 



The case 1 model is. used in this study for several reasons* 
First, the study is based on the results of a survey of randomly 
selected faculty/ administrators, and trustees in a sample of four 
year colleges* Second, most of the items in the survey measured 
unique constructs — such as morale, conflict/ student^f acuity 
relations/ etc. The case 2 irodel is generally inappropriate in 
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this instance* since we would be assessing agreement with respect 
to more than one construct or dimension. Third, by adopting thr 
case 1 model, we obtain an index of the extent to which raters 
agree about specific conditions within each of their respective 
organizations* irrespective of differences between organizations* 



The Nature of Organizational Climates 

In their paper on the etiology of organizational climates* 
Schneider and Reichers (1983) argue that there are two 
methodological issues related to the measurement of climates that 
are unresolved. One, concerns the aggregation problem discussed 
in the introduction to this paper. The other concerns the 
multidimensional nature of the climate construct. 

The multidimensional issue centers around where one should draw 
the boundaries for the climate construct, Glick (1985) suggests 
that it is difficult to distinguish between the domain of 
organizational climate and organizational "structural 
characteristics", such as technology, formalization, 
communication, managerial function, etc, James (1982) , on the 
other hand, argues that the issue is not what characteristics are 
included within the construct, but simply whether the 
characteristics are open to interpretation. 

In some ways, what constitutes the dimensions of organizational 
climate is incidental to the aggregation issue. That is, 
irrespective of the characteristics we believe legitimately 
constitute the construct, we are still confronted with the 
validity of aggregating the data we collect to more macro units of 
analysis. 

In other ways, however, the dimensions issue is quite important. 
From our perspective, the critical question is whether structural 
characteristics have the property of objective, "bedrock reality" 
{Ashworth, 1985)* Or, whether they are as subjective as any other 
"psychological variable (James, 1982) — e.g, , autonomy, equity, 
influence, support, warmth." If structural variablesf are just a 
class of psychological variables, then the whole issue of drawing 
the boundaries becomes mute* 

While this study Is not directly concerned with this problem, it 
sheds light on the issue by virtue of the organizational 
characteristics that are assessed in the work that follows. 



Research Method 



The analysis focuses on assessing the extent to which data 
collected in a national study meet the internal consistency 
criterion for aggregating perceptual data to the institution 
level. The national study was based on the development and 
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administration of a survey that included questions specifically 
designed to assess key dimensions of institutional performance and 
effectiveness. All questions asked respondents to take an 
institutional, rather than an individual focus. The instrument is 
shown in Appendix 1. 

Most items in the survey, with the exception of those in Section 
5 and 8, were constructed so as to measure different constructs. 
Section 5 items differ both in structure and nature from all other 
items in the survey. They are intended to measure homogeneous 
constructs, and are therefore more appropriately assessed using 
the Case 2 model. Research examining perceptual agreement of the 
items in Section 5 is reported in Krakower (1987). 

The items in Section 8 were intended to measure homogeneous 
constructs. However, the results of factor analysis provide 
marginal support for the item-construct relationships , Hence , the 
items in Section 8 are separately analyzed in this study. 

The construct measured by each item is reported in Table 1. 
Complete descriptions of each construct as well as other 
psychometric properties of the survey instrument are reported by 
Krakower and Niwa (19B5). 

[Table 1 about here] 

Colleges selected for participation in the national study were 
stratified on the basis of enrollment size (200 to 20,000 
students); institutional control (public versus private); the 
presence or absence of graduate programs; and enrollment changes 
between 197B and 1981 (growing, stable, declining). 

717 institutions were invited to participate in the study. 334 
institutions agreed to participate, received, and returned 
questionnaires . Table 2 details the participant institution 
characteristics according to the four selection characteristics. 

[Table 2 about here] 

A contact person was designated by the president of each 
institution. The contact person provided the names of trustees, 
administrators, and randomly selected faculty.* The survey was 
sent to a random sample of trustees, faculty, and key 
administrators in each institution. On average, 21 surveys were 
sent to each insti tution — seven to trustees , seven to 
administrators, and seven to faculty. 

The number of respondents per insti titution ranged from one to 
nineteen. Ninety-three percent of the institutions had seven or 
more respondents. The overall response rate was approximately 
48^. 3,406 people responded to the survey — 1,321 administrators 
(39^), 1,15B faculty (34*), and 927 trustees (27^). The average 
per institution was 4.0 administrators, 3.5 faculty, and 2.8 
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trustees. The overall average was 10.2 respondents per 
institution. 

Resul ts 

Intraclass correlations were calculated for all items in the 
survey with the exception of the items in Section 5. (Section 5 
intraclass correlations are reported elsewhere. } These are 
reported for administrators, faculty, and trustees in each 
institution, and for combined respondents in each institution in 
Table 3. 

[Table 3 about here] 

By way of example, the first line in the table shows intraclass 
coefficients of .07, .06, .00, and ,04, for administrators, 
faculty, trustees, and combined respondents in each institution, 
respectively, for item 1 in Section 1. The item reads: 

Major factors outside our institution that affects its 
enrollments have become more predictable over the past few 
years . 

The small magnitude of the coefficients in this example indicate 
that, in general, there is essentially no agreement among raters 
Kithin each of the institutions included in this study with 
respect to major factors affacting enrol Jiaents- 

The coefficients reported in Table 3 range between .00 and .60. 
However, more than Q0% of the coefficients are less than .30- The 
results are consistent across administrator, faculty, and trustee 
groups . 

The items in sections 4, 6, and 7, on the survey dealt almost 
entirely with institutional structural characteristics — e.g. , 
specialization, formalization, centralization, planning , 
diversity, resource allocation, etc. The intraclass correlations 
for the structural items are, in general, no higher than for more 
^^psychologically*' oriented items on the survey. The results 
stron9ly suggest that perceptions of organizational structural 
characteristics are clearly open to individual perceptual 
interpretation- 



Implications 

The results of this research strongly argue against aggregating 
individuals perceptions of organizational conditions to more macro 
units of analysis. The results indicate that organizational 
climate scores that are derived by averaging individuals' 
perceptions of specific organizational conditions do not reflect 
how individuals in an organization generally perceive, understand, 
or respond to organizational conditions. 
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The results suggest that employing group means to draw conclusions 
about organizational conditions may lead to spurious conclusions. 
It also follows that future research based on aggregated 
perceptual data must assess and demonstrate perceptual agreement 
before drawing any conclusions based on such data. The results 
also suggest th^t the reliability and validity of previous studies 
In higher education that draw conclusions based on aggregated 
perceptual data may be seriously flawed. 
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Table 1 

Constructs Measured by the Survey 



Section 



Item 



Construct 



Changes In the institution's 
external environment 



1 

2 
3 
4 

5 
6 
7 

a 



Enrollment Predictability 
Revenue Predictability 
Competitor Predictability 
Students' Tastes & Preferences 
Intensity of Competition 
Enrollment Competition 
Supply of students 

Availabilicy of Financial Resources 



Decreasing Enrollments 



1 

3 
4 

5 
6 



Consensus 

Inevitability 

Threat 

Administrative Control 
Duration 



1 

3 
4 

5 
6 



Decreasing Revenues 

Consensus 

Inevitability 

Threat 

Administrative Control 
Duration 



Institutional Characteristics 



1 

2 

3, 
7 

a 

9 

10 

11 

12 
13 
14 
IS 
16 
17 

la 



Specialization 

Formalization 

Mission 

Investor Confidence 

Structural Coupling 

Centralization 

Planning 

Innovation 

Scapegoating 

Resistance to Change 

Administrative Turnover 

Morale 

Slack Resources 
Interest Groups 
Administrator Credibility 
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17 



19 
20 
21 
22 



Reallocation Priorities 
Conflict 

Locus of Control 
Internal Mobility 



1. 
4, 



2 
7 



5, 8 

6, 9 

3, 10, 11, 
12, 13, 14 



Institutional Strategy 

Diversity 
Conservatism 
Moderate Change 
Innovation 

Administration 



1, 7 

2, 8 

3, 9 

4, 10 

5, 11 

6, 12 



Institutional Decision Processes 

Bureaucratic Allocation 
Autocratic Allocation 
Cdllegial Allocation 
Rational Allocation 
Allocation as Organized Anarchy 
i^olitical Allocation 



1, 8, 10 
12, 13, 14 
15, 16, 17, 
22, 23, 24 

3, 4, 25 
5, 6, 7 

18, 19, 20, 21 

26, 27, 28, 
29, 30, 31, 32 

2, 9, 11 



Performance and Actions of the 
Institution 

Student Educational Satisfaction 
Student Academic Development 

Professional Development & Quality 

of the Faculty 
Student Personal Development 
Faculty & Administrator Employment 

Satisfaction 
System Openness & Community 

Interaction 

Organizational Health 
Student Career Development 
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Table 2 

Number of Institutions in Study Sample 
by Selection Criteria 
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Table 3 
Intraclass Correlations 



Aiinlnlstrators Faculty Trustees Total 
Section item n=1321 ifIISS n=927 ry=3406 



1 

2 
3 
4 
5 
6 
7 
8 



.07 
.04 
.00 
.04 
.12 
.17 
.22 
.23 



.06 
.08 
.01 
.02 
.14 
.15 
.16 
.14 



.00 
.03 
.00 
.10 
.10 
.14 
.18 
.22 



.04 
.05 
.01 
.03 
.13 
.16 
.17 
.17 



2 1 .60 

3 .19 

4 .31 

5 .14 

6 .31 

3 1 .31 

3 .14 

4 .28 

5 .00 

6 .32 

4 1 .17 

2 .14 

3 .30 

4 .28 

5 .17 

6 .24 

7 .19 

8 .06 

9 .12 

10 .25 

11 .13 

12 .16 

13 .21 

14 .35 

15 .23 

16 .23 

17 .15 

18 .20 

19 .10 

20 .22 

21 .25 

22 .50 



.55 .38 .49 

.08 .17 .14 

.28 .34 .28 

.14 .10 .11 

.25 .28 .28 

.29 .29 .29 

.12 .09 .09 

.29 .17 .22 

.12 .00 .09 

.11 .24 .Id 

.16 .10 .14 

.11 .08 .10 

.31 .18 .26 

.29 .23 .24 

.18 .13 .14 

.23 .14 .19 

.12 .06 .11 

.06 .06 .07 

.11 .06 .08 

.22 .22 .18 

.13 .12 .10 

.07 .14 .10 

.15 .16 .13 

.38 .26 .30 

.23 .20 .16 

.07 .12 .12 

.06 .14 .11 

.23 .14 .14 

.16 .07 .06 

.25 .21 .17 

.12 .13 .16 

.55 .37 .45 



6 1 .24 

2 .21 

3 .19 

4 .09 

5 .18 

6 .24 



.16 .13 .18 

.12 .06 .15 

.10 .10 .10 

.13 .10 .06 

.20 .09 .13 

.28 .21 .21 
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Dear Respondent: 



This questionnaire is part of a national study of performance in colleges and univer- 
sities conducted by the National Center for Higher Education Management Systems, 
Several administrators, faculty department heads, and trustees at your institution are 
completing this instrument* You were selected as a respondent because of the posi- 
tion you hold at this school. 

We are seeking your perceptions of the overall institution rather than information 
about one particular department or program. The responses of all individuals uill 
remain strictly confidential The data will be analyzed at MCHEMS in Boulden 
Colorado, and all individual responses will be aggregated. In addition, the name of 
your institution will be revealed only to individuals at your school in the feedback 
reports to be provided at the conclusion of the study. You will be able to compare 
your institution with other similar schools, but the other schools will be described on 
the basis of their general characteristics, not by name. 

The questionnaire is designed to be mailed back to MCHEMS without needing an 
envelope. On the back cover is printed the address of MCHEMS, along with a sticker 
identifying your institution as the return address. Just seal up the questionnaire and 
drop it in the maiL We will pay the return postage. You wll find three peel-off stickers 
included with the questionnaire for your use in sealing up the qu^:stionnaire prior to 
mailing it. 

Please complete the questionnaire at your earliest convenience; if possible, we 
would like it within 10 days of when you received it. Previous respondents have 
averaged 20 minutes to complete the questionnaire, so despite its length, we hope 
you find the questions interesting and thought-provoking. If you have questions or 
comments, please feel free to contact Dr Kim Cameron at (303) 497-0368. Thank 
you in advance for your cooperation- 
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SECTION 1: Changes in the institution's External Environment! 



The following questions concern changes En conditions outside your institution over 
the past few years. Please i:ircle the number to the right of each statement that 
best reflects your institution's experiences since 1979-80* 




1 . Major factors outside our institution that affect its enrollments have become more 
predictable over tlie past few years. 


1 


2 


3 


4 


5 


-10 


2, Major factors outside the institution that affect its revenues liave become less 
predictable over the past few years. 


1 


2 


3 


4 


5 


- 1 1 


3. Competitive actions of other colleges and universities have become more 
predictable over the past few years. 


1 


2 


3 


4 


5 


2 


4. The tastes and preferences of students have become harder to forecast over the 
past few years. 


1 


2 


3 


4 


5 


~ 1 J 


5. Competitive actions of other colleges and universities now affect this institution in 
more areas (e.g., price, programs, area served) than in the past. 


1 


2 


3 


4 


5 


— 1 J 


6. Competition with other colleges and universities for student enrollments has 
increased over the past few years. 


1 


2 


3 


4 


5 


_13 


7. The number of potential students from whom our institution can recruit has 
increased over the past few years. 


1 


2 


3 


4 


5 


-16 


8. Financial resources have become more difficult to obtain over the past few years. 


1 


2 


3 


4 


5 


_I7 



sSECTION 2: Decreasing Enrollments! 



This section is concerned with whether your institution has experienced decreasing 
fulHime equivalent enrollments during any of the academic years since 1979-80* 

1. To ihe best of your (knowledge, did full-time equivalent student enrollments 
decrease from one year to ihe next during any of the academic years from 1 979-80 
to 1982-83? 

If you answered "no" to ihe above question, please sitip to Section 3 on the following 
page. If you answered "Ves" please complete the remaining itenre in this section. 

2. Please checlt the years in which you believe thai full-lime equivalent enrollments 
decreased from those of the previous year 



1979-80 



1980-81 



1981-82 



Please circle the number to the right of each statement that best reflects your insti- 
tution's experiences during its most recent episode of decreasing enrollments. 

3. Decreasing enrollments were inevitable al ihat lime. 

4. Decreasing enrollments presented an immediale threat to ihe viability of this 
institulion. 

5. Prediclions of decreasing enrollments provided adequate lead time to lake actions 
that minimized their impact. 

6. Decreasing enrollments were a short-term problem. 

7. Please indicate in the space below ihe major factors that caused enrollments lo 
decrease at your institution. 



( 1 ) Yes 

(2) Mo 



1982-83 




1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 
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-20 at 

22 23 
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-2& 



.26 
-27 



-2B29 
30 31 
32 33 



SECTION 3: Decreasing Revenues 



This section is concerned with whether your institution has experienced decreasing 
revenuest adjusted for inflation, during any of the academic years since 1979-80^ 

1 , To Che best of your knowledge, did revenues, adjusted for inflation, decrease from 
one year to the next during any of the academic years from 1979-80? 

If you answered "no" to the above question, please skip to Section 4, which begins on 
this page. If you answered 'yes," please complete the remaining items in this section. 

2, Please check the years in which you believe that revenues, adjusted for inflation, 
decreased from those of the previous year, 

1979-80 1980-81 1981-82 _ 



( 1 ) Yes 

(2) Mo 



1982-83 



-35 



-36 37 
35 J9 



Please circle the number to the nght of each statement that best reflects your insti- 
tution's experiences during its most recent episode of decreasing revenues* 



3, Decreasing revenues were inevitable at that time, 

4, Decreasing revenues presented an immediate threat to the viability of the 
institution. 

5, Predictions of decreasing revenues provided adequate lead time to take actions that 
minimized their impact, 

6, Decreasing revenues were a short-term problem, 

7, Please indicate in the space below the major factors that caused revenues to 
decrease at your institution. 




1 


2 


3 


4 


5 


1 


2 


3 


4 


5 






* 






1 


2 


3 


4 


5 


1 


2 


3 


4 


5 



-40 



-42 
—43 



^■45 
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tSECTlON 4: Institutional Characteristics 



In this section, we are asking for your impressions of some general characteristics 
of your institution* Please answer each item* If you are not suret make your best 
guess. 




1. This institution has many administrators performing specialized functions. 


1 


2 


3 


4 


5 


-5t 


2. Formal policies and rules govem most activities at this institution. 


1 


2 


3 


4 


5 


-52 


3. This institution has a special identity, unlike any other in higher education. 


1 


2 


3 


4 


5 


_53 


4. There is a general sense that this institution has a distinctive purpose to fulfil. 


1 


2 


3 


4 


5 


-54 


5. The academic programs offered here reflect the mission of the institution. 


1 


2 


3 


4 


5 


-55 


6. People associated with this institution share a common deRnition of its mission. 


1 


2 


3 


4 


5 


_56 


7. Those who make a personal or financial investment in this institution believe that 
they receive an ample retum. 


1 


2 


3 


4 


5 


-57 


8. The activities of the various units in this institution are loosely coordinated or loosely 
coupled. 


1 


2 


3 


4 


5 


-S8 


9. Major decisions are very centralized. 


1 


2 


3 


4 


5 


-59 


10. Long-term planning is neglected. 


1 


2 


3 


4 


5 


-60 
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Institutional Characteristics (continued) 




1 1 . Innovative activity is increasing. 


1 


2 


3 


A 


5 


'^6t 


1 2- Top administrators are often scape goats. 


I 


2 


J 


A 


c 
D 


-62 


1 3- There is a lot of resistance to change in this schooi. 


1 


2 


J 


A 

A 


c 
D 


-63 


14. There is a great deal of turnover in adminibtrative positions. 


1 


2 


J 


A 

A 


D 


-6-* 


15- Morale is increasing anrK:>ng members of this institution. 


1 


2 


J 


A 

A 


c 
D 


-6b 


16^ We have no place that we could cut expenditures without severely damaging the 
school. 


1 


2 


J 


A 

A 


c 
D 




17. Special interest groups within the institution are becoming more vocal. 


\ 


2 


3 


A 


5 




18. Top administrators ha\e high credibility. 


1 


2 


3 


A 

4 


5 


-68 


19. When cutbacks occur, they are done on a prioritized basis. 


1 


2 


3 


4 


5 




20. Conflict is increasing within this institution. 


1 


2 


3 


4 


5 


-70 


21. Top administrators believe that factors outside the institution largely determine its 
condition. 


1 


2 


3 


4 


5 




22. Top administrative positions are now held by individuals who were promoted fron 
within the institution. 


1 


2 


3 


4 







iSECTION 5* Type of Institution- 



These questions relate to the type of organization that your institution Is most like* Each of these Items con- 
tains four descriptions of institutions of higher education. Please distribute 100 points among the four descnp* 
tions depending on how similar the description is to your school* None of the descriptions is any better than 
the others; they are just different For eacft quesUorif please use a// 700 points, 

FOR EXAMPLE: 

In question 1, if institution A seenre very similar to mine, B seems somewhat similar, and C and D 
do not seem similar at all, 1 might give 70 points to A and the remaining 30 points to B. 



1. Institutional Characteristics (Please distribute 100 points). 

Institution A is a very personal place. It is like 

an extended family. People seem to share a lot 
of themselves. 



points 
for A 



Institution C is a very formalized and struc- 
points tured place. Bureaucratic procedures gen- 
erally govern what people do. 



forQ 



Institution B is a very dynamic and entrepre- 
neuriaf place. People are willing to stick their 
necks out and take risks. 



points 
for B 



Institution D is very production oriented. A 

9<^*^^ major concern is with getting the job done, 
for D People aren't very personally involved. 



,74,75 

76,77 
76.79 

60 e I 



2. Institutional Leader (Please distribute 100 points) 
The head of institution A is generally consid- 



points 
for A 



points 

RIC 



ered to be a mentor, a sage, or a father or 
mother figure. 

The head of institution C is generally consid- 
ered to be a coordinator, an organizer, or an 
adntinlstrator. 



The head of institution B is generally consid- 
ered to be an entrepreneur, an innovator, or 
a risk taken 



points 
for B 



points 
for D 



The head of institution D is generally consid* 
ered to be a producer, a technician, or a hard- 
driver* 
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Type of Institution (continued) 



3. Institutional '*Olue" (Please distribute 100 points) 

The glue that holds institution A together is 

points loyalty and tradition. Commitment to this 
^^^^ school runs high. 

The glue thai holds institution C together is 

P^'^t^ formal rules and policies. Maintaining a 
^^^^ smooth running institution is important here 



The glue that holds institution B together is a 

points commitm*;nt to innovation and develop* 
^ ment There is an emphasis on being first. 

The glue that holds institution D together is the 

points emphasis on tasks and goal accomplishment 
^ A production orientation is commonly shared. 



4. Institutional Emphases (Please distribute 100 points) 

Institution A emphasizes human resources. 

High cohesion and nrorale in the school are 
important. 

Institution C emphasizes permanence and 
stability. Efficient, smooth operations are 
important. 



points 
for A 



poinls 
forC 



Institution B emphasizes growth and acquir- 
ing new resources. Readiness to meet new 
challenges is important. 

Institution D emphasizes competitive actions 
and achievement. Measurable goals are 
important. 



points 
for B 



points 
forD 



-96 99- 
102 103 



! SECTION 6: Institutional Strategy 



The following section deals with the strategy your institution is pursuing. Please 
indicate the extent to which you agree or disagree with each itemi based on your 
own perceptions. 




ERIC 



1 . We are making our academic programs more diverse. 


1 


2 


3 


4 


5 


-107 


2. We are changing the comfxasition of our student body, making it more diverse. 


1 


2 


3 


4 


5 


-106 


3. We are increasing the investment of the college in functions that deal with external 
people (admissions, development, government relations, and others). 


1 


2 


3 


4 


5 


-109 


4. This institution tries to insulate itself from pressures in the environment. 


1 


2 


3 


4 


5 


-110 


5. This institution tries new activities or policies, but not until after others have found 
them successful, 


1 


2 


3 


4 


5 


-111 


6. This institution is likely to be the first to try new activities or policies. 


t 


2 


3 


4 


5 


-112 


7, Our top administrators educate important outsiders about the value of the institu- 
tion in order to improve its legitimacy in their eyes. 


1 


2 


3 


4 


5 


-n3 


8, This institution tends to do more of what it does well, to expand in areas we have 
expertise. 


1 


2 


3 


4 


5 


-114 


9. This institution establishes new domains of activity. 


1 


2 


3 


4 


5 


-lis 


10. We are increasing the quality of the individuals in top administrative positions. 


1 


2 


3 


4 


5 


— 116 


1 1 . Top administrators emphasize finding new money, more so than saving money, for 
a balanced budget. 


1 


2 


3 


4 


5 


— 117 


12. The top administrative team has developed multi-year stategies to achieve long- 
term institutional objectives. 


1 


2 


3 


4 


5 


-118 


1 3. The top administrative team receives rapid and accurate feedback about enrollment 
and financial conditions. 


1 


2 


3 


4 


5 


-119 


1 4. The top administrative team provides incentives for conserving resources. 


1 


2 


3 


4 


5 


-120 
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Institutional Strategy (continued) 



15. Of the four actions listed below, which one is the most likely response of this institution to changes in the 
outside world? (check one response) 

1. Change the institution's policies and procedures 

2. Change the institutions inriage through communication 

3. Change the kinds of students, suppliers, or donors we deal with 

4. Weather any storm, making no changes 

16. Of the four actions listed below, which one is the least likely response of this institution to changes in the 
outside world? (check one response) 

1 . Change the institution "s policies and procedures 

2. Change the institution "s image through communication 

3. Change the kinds of students, suppliers, or donors we deal with 

4. Weather any storm, making no changes -^22 



t SECTION 7: Institutional Decision Processes 



The following questions deal with the decision process used at the Institution for 
allocating resources— whether the resources are staff positions, dollars, space, or 
other valuable items. Please indicate the extent to which you agree or disagree with 
each item* 





1 . This institution has a standard set of procedures it uses to make resource allocation 
decisions. 


1 


2 


3 


4 


5 


— 124 


2. One individual at this iristitution makes all resource allocation decisions of any 
consequence. 


1 


2 


3 


4 


5 


-125 


3. People at this Institution make resource allocation decisions collegialty. 


1 


2 


3 


4 


5 


-126 


4. A rational process is used to make resource allocation decisions at this institution. 


1 


2 


3 


4 


5 


-127 


5. No particular (jattem characterizes the process by which resource allocation 
decisioris are made here. 


1 


2 


3 


4 


5 


-128 


6. Resource allocation decisions are political, based on the relative pOfwer of those 
Involved. 


1 


2 


3 


4 


5 


-129 


7. Resource allocation is decided bureaucratically at this Institution. 


1 


2 


3 


4 


5 


-1KI 


8. Resource allocation is decided autocratically. 


1 


2 


3 


4 


5 


-I3l 


9. Resource allocation is a matter for group discussion and consensus. 


1 


2 


3 


4 


5 


-132 


10. Resource allocation deci^oris are based on what objectively seems best for this 
Institution overall. 


1 


2 


3 


4 


5 


-133 


1 1 . Resource atlocation Is decided by coincidence: it is a matter of organized anarchy. 


\ 


2 


3 


4 


5 


— 134 


12. Persuasion, negotiation, and coalition-building are examples of what determines 
resource allocation. 


1 


2 


3 


4 


5 


-135 
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SECTION 8: Performance and Actions of the Institution 



The items in this section ask about the performance and actions of your institution* 
If you are not sure of the ttem* please make your best guess. 

To what extent are the following characteristics typical of thts institution? 




1* One of the outstanding features of this institution is the opportunity It provides stu* 
dents for personal development in addition to academic development. 


1 


2 


3 


4 


5 


-137 


2* This college is highly responsive and adaptive to meeting the changing needs of its 
external constituencies. 


1 


2 


3 


4 


5 


_13B 


3, This college has a very high ability to obtain financial resources in order to provide a 
high quality educational program. 


1 


2 


3 


4 


5 


-139 


4, When hiring new faculty members^ this college can attract the leading people in the 
country in their respective fields to take a job here. 


1 


2 


3 


4 


5 


-UO 


5. There seems to be a feeling that dissatisfaction is high among students at this 
institution 


1 


2 


3 


4 


5 


-141 


6. There have been relatively large numbers of students either drop out or not return 
because of dissatisfaction with their educational experiences here. 


1 


2 


3 


4 


5 


-142 


7, 1 am aware of a large number of student complaints regarding their educational 
experience here as registered in the campus newspaper, meetings with faculty 
members and administrators, or other public forums. 


1 


2 


3 


4 


5 


-143 


8. There is a very high emphasis on activities outside the classroom designed specif- 
ically to enhance students* personal, non-academic development. 


1 


2 


3 


4 


5 


— 144 


9. There is a very high emphasis on institution-community or institution-environment 
activities. 


1 


2 


3 


4 


5 


-145 


10, Students develop and mature in non-academic areas (e.g., socially, emotionally, 
culturally) to a very large degree directly as a result of their experiences at this 
institution. 


1 


2 


3 


4 


5 


-146 


1 1. A very targe number of community-oriented programs, workshops, projects, or 
activities were sponsored by this institution last year. 


1 


2 


3 


4 


5 


-147 



1 2, Think of last years graduating class at this institution. Please rate the academic attainment or academic level 
achieved by that class as a whole, (Select one) 



1) That class is among the very top classes 
in the country, 

. 2) That class is well above average* 

. 3) That class is slightly above average, 

4) That class is about average. 



5) That class is slightly below average, 

6) That class is below average. 

7) That class is near the bottom of 
classes across the country. 



-ue 



1 3, Estimate what percent of the graduates from this institution go on to obtain degrees tn graduate or professional 
schools. 



ERLC 



. 1) From 91 % to 100% of the students 
here go on for advanced degrees, 

2) From 76% to 90% go on. 

3) From 61% to 75% goon, 

4) From 46% to 60% go on. 
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5) From 31 % to 45% go on, 

6) From 16% to 30% go on, 

7) From 0 to 15% goon to obtain 
advanced degrees. 



,-149 



! Performance and Actions of the Institution (continued)! 



Please use the following scale in responding to the following question? 

7 — All 5 — More than half 3 — Less than half 



1 — Mone 



6 — A large majority 



4 — About half 



2 — A small minority 



14.. 

15., 

16., 

17., 

18.. 

19.. 

20.. 

21., 
22.. 

23. 

24., 
25. 



How many students would you ^^ay engage in extra academic work (e.g.. reading, studying, writing) 
over and above what is specifically assigned in the classroom. 

What proportion of the students who graduated from this institution last year and ntered the labor 
market obtained employment in their major field of study? 

How many students would you say attend this college to fulfill definite career or occupational goals 
as opposed to attending for social athletic^ financial or other reasons? 

Of those students who obtained employment after graduating from this institution, for how many of 
them was career training received at this institution important in helping them obtain their jobs? 

If given the chance of taking a similar job at another school of his or her choice, how many faculty 
members do you think would opt for leaving this school? 

If given the chance of taking a similar job at another school of his or her choice, how many adminis- 
trators do you think would opt for leaving this school? 

Estimate how many faculty members at this institution are personally satisfied with their 
employment. 

Estimate how many administrators at this college are personally satisfied with their employment. 

How many faculty members at this institution would you say published a book or an article in a 
professional journal or displayed a work of art in a show last year? 

What proportion of the faculty members would you estimate teach at the "cutting edge* of their 
field — i.e*, require current journal articles 3S reading, revise syllabi at least yearly, discuss current 
issues in the field, etc*? 

How many faculty members at this college are actively engaged now in professional development 
activities— e.g., doing research, getting an advanced degree* consulting^ etc.? 

Colleges may be rated on the basis of their relative 'drawing power'* in attracting top high school 
students. In relation to other colleges with which it competes^ what proportion of the top students 
attend this institution rather than the competition? 



-ISO 



-132 



-155 



-t36 
*157 



_l5a 



-159 



This section asks you to rate your perceptions of the general day*to-day functioning of the overall institution. Please 
respond by circling the number that best represents your perceptions of each item. If you agree strongly with one 
end of the scale, circle a number closer to that end of the scale. If you feel neutral about the item, circle a number 
near the middle of the scale. 



FOR EXAMPLE: 

How is the weather in this town? 
warm, bhght. and sunny 

How do you perceive the following? 

26* Student/ faculty relationships 
unusual closeness, tots of informal 
interaction, mutual personal concern 



©3 



4 5 6 7 



1 2 3 4 5 6 7 



cold, wet, and dismal 



no closeness, mostly instrumental 
relations, little informal interaction 



-162 
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27. Equity of treatment and rewards 
people treated fairly and 
rewarded equitably 



1 2 3 4 5 6 7 

30 



favoritism and inequity present, 
unfair treatment exists 



Performance and Actions of the Institution (continued) 



28. Organizational health of the college 
college njns smoothly, healthy 
organization, productive internal 
functioning 

29. General levels of trust among people here 
high suspicion, fear distrust, 
insecurity 

30. Conflicts and friction in the college 
large amount of conflict disagree* 
ments, anxiety, friction 

31. Recognition and rewards received for good work from superiors 
recognition received for good ^ ^ ^ & c -7 
work, rewarded for success 



1 2 3 4 5 6 7 



12 3 4 5 6 7 



1 2 3 4 5 6 7 



32. The amount of information or feedback you receive 
feel informed, in^the know, 
information is always available 



college runs poorly, unhealthy 
organization, unproductive internal 
functioning 



1 2 3 4 5 6 7 



high trust, security, openness 



no friction or conflicts* friendly, 
collaborttive 



no rewards for good work, no one 
recognizes success 

feel isolated, out^of^it. 
information is never available 



-166 



! SECTION 9: Respondent Demographics: 



These Items ask for some personal demographic information* This information will not t>e used to try to 
identify yout rather it simply will help us In our analysis of the questionnaire data* Please answer each item* 

L In what year were you bom? lil 

-174 

2. In how many organizations have you worked in your professional career? _ 

_J76 

177 



3. How many years have you held your current position? 



4. Are you male or female ? _i7e 

5. Have you received degrees (i.e., bachelors, masters, or doctorate) in any of the following fields? (please 
check all that apply) 

1) Business administration 4) Health Care administration 

2) Educational administration 5) Personnel or Industrial administration 

3) Public administration 6) Oiher adminisiraiion fields -179 



6, In what field did you receive your last degree? 

1) Humanities (e.g.. literature, languages) 6) Mathematics and Computer Sciences 

2) Fine Arts (e.g., music, sculpture) 7) Professional Fields (e.g., law. 

3) Physical Sciences (e.g., physics, chemistry) engineering) 

4) Biological Sciences (e.g., zoology, botany) 8) Administration Fields (educational. 

5) Social Sciences (e.g., sociology business) 

economics) 9) Other 



- ISO 



7. How many years have you been affiliated with this institution? -tei 

162 

8. What is your highest academic degree? 

1 ) Doctorate or other terminal degree 

2) Masters 

3) Bachelors 

4) Associate ^i8j 

o 31 
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